feat: remove Python 3.7 support #5191

rwgk · 2024-06-24T17:03:20Z

Description

Remove Python 3.7 support entirely. (Official end-of-life: 2023-06-27)

NOTE: We're losing testing Clang versions 3, 7, 9, 10, and GCC version 7, but we're deliberately deferring reviewing the C++ code to remove related #ifdefs.

Suggested changelog entry:

Support for Python 3.7 was removed. (Official end-of-life: 2023-06-27)

…/pybind11/pull/5177/commits

…th Python 3.7

rwgk · 2024-06-24T20:16:37Z

Overview of all compiler versions used/lost(-)/gained(+) before this PR, and with this PR @ commit 2f24de5:

Clang release dates: https://releases.llvm.org/
GNU release dates: https://gcc.gnu.org/releases.html

 AppleClang 15.0.0.15000040
-Clang 3.6.2    16 Jul 2015
-Clang 3.7.1     5 Jan 2016
-Clang 3.9.1    23 Dec 2016
 Clang 5.0.2
-Clang 7.1.0    10 May 2019
-Clang 9.0.1    20 Dec 2019
-Clang 10.0.1    6 Aug 2020
 Clang 11.1.0
 Clang 12.0.1
 Clang 13.0.1
 Clang 14.0.6
 Clang 15.0.7
 Clang 16.0.6
 Clang 18.1.6 with GNU-like command-line
 Clang 18.1.7
 Clang 19.0.0
-GNU 7.5.0      November 14, 2019
-GNU 8.3.0      February 22, 2019
 GNU 8.5.0
 GNU 9.4.0
+GNU 10.2.1
 GNU 10.5.0
 GNU 11.4.0
 GNU 11.4.1
 GNU 12.4.0
 GNU 13.2.0
 GNU 13.2.1
 GNU 13.3.0
 Intel 2021.10.0.20230609
 MSVC 19.29.30154.0
 MSVC 19.40.33811.0
 NVHPC 23.5.0

rwgk · 2024-06-24T22:18:45Z

Tracking simple findings:

ubuntu-18.04 apt-get update && apt-get install -y gcc produces g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, BUT ubuntu-18.04 is not longer available: The Ubuntu 18.04 Actions runner image will begin deprecation on 2022/08/08 and will be fully unsupported by 2023/04/03 actions/runner-images#6002
ubuntu-20.04 apt-get update && apt-get install -y clang produces clang version 10.0.0-4ubuntu1

This reverts commit 518515a.

Revert "Try running in ubuntu:18.04 container under ubuntu-latest (to get GCC 7)" This reverts commit b203a29.

Removes two changes made under pybind#3702

This reverts commit bb5b9d1.

``` ... -- The CXX compiler identification is Clang 15.0.7 ... - Found Python: /usr/bin/python3.9 (found suitable version "3.9.2", minimum required is "3.7") found components: Interpreter Development.Module Development.Embed ... /__w/pybind11/pybind11/include/pybind11/gil.h:150:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = internals.tstate; ^~~~~ auto * /__w/pybind11/pybind11/include/pybind11/gil.h:174:13: error: 'auto key' can be declared as 'auto *key' [readability-qualified-auto,-warnings-as-errors] auto key = detail::get_internals().tstate; ^~~~~ auto * ```

henryiii · 2024-06-26T03:10:52Z

I think this next release might be a good time to remove C++11 support and increase our minimum supported compilers.

rwgk · 2024-06-26T03:15:06Z

I think this next release might be a good time to remove C++11 support and increase our minimum supported compilers.

Awesome. That's what I was thinking, too. I'll work on this PR more after the 2.13 release has baked in a bit.

henryiii · 2024-06-26T19:50:54Z

I think we can also bump CMake support. Personally, I've been going 3.15+ everywhere as that is what was required to build scikit-build-core. Though another option is 3.12+ (FindPython). I think we has someone request 3.10 or 3.11 before, might be worth checking.

If we did go 3.15+, we maybe could start to phase out classic FindPythonInterp support.

rwgk · 2024-06-26T22:03:23Z

If we did go 3.15+, we maybe could start to phase out classic FindPythonInterp support.

I just looked up: CMake 3.15.0 publication date was 17 July 2019

Does that look correct? — That's already 5 years ago (almost at least), and before Python 3.8 was released (Oct 2019). — I think it's much more likely to be helpful than harmful to make CMake 3.15 the minimum, and shed cruft.

henryiii · 2024-06-27T03:52:24Z

Here's the support table for CMake, with darker red for older than 3.15:

rwgk · 2024-06-27T06:01:53Z

My thinking:

I want to wait about one week, JIC we need more 2.13 patch releases.
Then I'll make a fresh pass through this PR, looking for 3.7 and 3.8 version numbers of all sorts once more.
Then merge this PR, documenting in the PR description that we're losing testing with Clang 3.6 through 10, and GCC 7 as a consequence of dropping Python 3.7 support, but we're deliberately deferring cleaning up ifdefs for another release.
Then have a follow-on PR to bump the cmake minimum to 3.15 (independently from dropping Python 3.7) and clean out cmake cruft.

henryiii · 2024-06-27T06:47:42Z

We’ve already made a 2.13 branch, master is 2.14. No rush, but waiting isn’t required. Does save ci time though if the branches are functionally identical.

No need to defer cleanup, we have plenty of time for this to sit in master. We should specify and test minimum versions where possible. I’m hoping we get a MSVC bump. :)

But that can also be a follow-up.

rwgk · 2024-06-28T14:29:48Z

but waiting isn’t required.

it more prioritizing than waiting

this is up high on my list though; not a lot more to do

…s below: ``` git_grep_37_38.sh |& sort | uniq -c ``` With git_grep_37_38.sh: ``` set -x git grep 0x0307 git grep 0x0308 git grep PY_MINOR_VERSION git grep PYPY_VERSION git grep -I -E '3\.7' git grep -I -E '3\.8' git grep -I -E '\(3, 7' git grep -I -E '\(3, 8' git grep -I -E '3[^A-Za-z0-9.]+7' git grep -I -E '3[^A-Za-z0-9.]+8' ``` Output: ``` 1 .appveyor.yml: $env:CMAKE_INCLUDE_PATH = "eigen-3.3.7;$env:CMAKE_INCLUDE_PATH" 1 .appveyor.yml: 7z x eigen-3.3.7.zip -y > $null 1 .appveyor.yml: Start-FileDownload 'https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip' 1 CMakeLists.txt: # Bug in macOS CMake < 3.7 is unable to download catch 1 CMakeLists.txt: elseif(WINDOWS AND CMAKE_VERSION VERSION_LESS 3.8) 1 CMakeLists.txt: if(OSX AND CMAKE_VERSION VERSION_LESS 3.7) 1 CMakeLists.txt: message(WARNING "CMAKE 3.7+ needed on macOS to download catch, and newer HIGHLY recommended") 1 CMakeLists.txt: message(WARNING "CMAKE 3.8+ tested on Windows, previous versions untested") 1 CMakeLists.txt: # Only tested with 3.8+ in CI. 1 docs/advanced/functions.rst:Python 3.8 introduced a new positional-only argument syntax, using ``/`` in the 1 docs/changelog.rst:* Adapt pybind11 to a C API convention change in Python 3.8. `pybind#1950 1 docs/changelog.rst:* Allow thread termination to be avoided during shutdown for CPython 3.7+ via 1 docs/changelog.rst: considered as conversion, consistent with Python 3.8+. 1 docs/changelog.rst: CPython 3.8 and 3.9 debug builds. 1 docs/changelog.rst:* Enum now has an ``__index__`` method on Python <3.8 too. 1 docs/changelog.rst: on Python 3.8. `pybind#1780 <https://github.com/pybind/pybind11/pull/1780>`_. 1 docs/changelog.rst:* PyPy 3.10 support was added, PyPy 3.7 support was dropped. 2 docs/changelog.rst:* Support PyPy 7.3.7 and the PyPy3.8 beta. Test python-3.11 on PRs with the 1 docs/changelog.rst:* Use ``macos-13`` (Intel) for CI jobs for now (will drop Python 3.7 soon). 1 docs/changelog.rst:* Use new Python 3.7 Thread Specific Storage (TSS) implementation if available. 1 docs/compiling.rst: cmake -DPYBIND11_PYTHON_VERSION=3.8 .. 1 docs/compiling.rst: find_package(Python 3.8 COMPONENTS Interpreter Development REQUIRED) 1 docs/limitations.rst:- PyPy3 7.3.1 and 7.3.2 have issues with several tests on 32-bit Windows. 1 docs/requirements.txt:idna==3.7 \ 1 + git grep 0x0307 1 + git grep 0x0308 1 + git grep -I -E '\(3, 7' 1 + git grep -I -E '3\.7' 1 + git grep -I -E '\(3, 8' 1 + git grep -I -E '3\.8' 1 + git grep -I -E '3[^A-Za-z0-9.]+7' 1 + git grep -I -E '3[^A-Za-z0-9.]+8' 1 + git grep PY_MINOR_VERSION 1 + git grep PYPY_VERSION 2 .github/workflows/ci.yml: - '3.8' 1 .github/workflows/ci.yml: - 3.8 1 .github/workflows/ci.yml: - name: Add Python 3.8 1 .github/workflows/ci.yml: - 'pypy-3.8' 2 .github/workflows/ci.yml: python: '3.8' 1 .github/workflows/ci.yml: - python: '3.8' 1 .github/workflows/ci.yml: - python: 3.8 1 .github/workflows/ci.yml: python: 'pypy-3.8' 1 .github/workflows/configure.yml: cmake: "3.8" 1 .github/workflows/configure.yml: name: 🐍 3.8 • CMake ${{ matrix.cmake }} • ${{ matrix.runs-on }} 1 .github/workflows/configure.yml: - name: Setup Python 3.8 1 .github/workflows/configure.yml: python-version: 3.8 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 & 📦 tests • ubuntu-latest 1 .github/workflows/pip.yml: name: 🐍 3.8 • 📦 tests • windows-latest 2 .github/workflows/pip.yml: - name: Setup 🐍 3.8 2 .github/workflows/pip.yml: python-version: 3.8 2 include/pybind11/cast.h:#if !defined(PYPY_VERSION) 2 include/pybind11/cast.h:#if defined(PYPY_VERSION) 2 include/pybind11/cast.h: // PyPy: 7.3.7's 3.8 does not implement PyLong_*'s __index__ calls. 5 include/pybind11/detail/class.h:#if !defined(PYPY_VERSION) 1 include/pybind11/detail/class.h:#if defined(PYPY_VERSION) 1 include/pybind11/detail/class.h: // This was not needed before Python 3.8 (Python issue 35810) 1 include/pybind11/detail/common.h: && !defined(PYPY_VERSION) && !defined(PYBIND11_ASSERT_GIL_HELD_INCREF_DECREF) 2 include/pybind11/detail/common.h:# error "PYTHON < 3.8 IS UNSUPPORTED. pybind11 v2.13 was the last to support Python 3.7." 1 include/pybind11/detail/common.h:#if defined(PYPY_VERSION) && !defined(PYBIND11_SIMPLE_GIL_MANAGEMENT) 1 include/pybind11/detail/common.h:#if PY_VERSION_HEX < 0x03080000 1 include/pybind11/detail/common.h: = PYBIND11_TOSTRING(PY_MAJOR_VERSION) "." PYBIND11_TOSTRING(PY_MINOR_VERSION); \ 1 include/pybind11/detail/internals.h: // called. PYBIND11_TLS_FREE is PyThread_tss_free on python 3.7+. On older python, it does 1 include/pybind11/detail/internals.h:#if PYBIND11_INTERNALS_VERSION <= 4 || defined(PYPY_VERSION) 1 include/pybind11/detail/internals.h:// The old Python Thread Local Storage (TLS) API is deprecated in Python 3.7 in favor of the new 1 include/pybind11/detail/type_caster_base.h:#if defined(PYPY_VERSION) 1 include/pybind11/embed.h:# define PYBIND11_PYCONFIG_SUPPORT_PY_VERSION_HEX (0x03080000) 1 include/pybind11/embed.h:#if defined(PYPY_VERSION) 1 include/pybind11/eval.h: // globals if not yet present. Python 3.8 made PyRun_String behave 2 include/pybind11/eval.h:#if defined(PYPY_VERSION) 2 include/pybind11/eval.h: // was missing from PyPy3.8 7.3.7. 2 include/pybind11/gil.h: /// allowed during shutdown. Check _Py_IsFinalizing() on Python 3.7+, and 1 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) 4 include/pybind11/pybind11.h:#if !defined(PYPY_VERSION) && PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION == 9 1 include/pybind11/pytypes.h:#endif //! defined(PYPY_VERSION) 2 include/pybind11/pytypes.h:#if !defined(PYPY_VERSION) 1 include/pybind11/pytypes.h:# if defined(PYPY_VERSION_NUM) && PYPY_VERSION_NUM < 0x07030a00 1 include/pybind11/pytypes.h:#ifdef PYPY_VERSION 1 include/pybind11/stl/filesystem.h:# if !defined(PYPY_VERSION) 2 pybind11/__init__.py:if sys.version_info < (3, 8): 2 pybind11/__init__.py: msg = "pybind11 does not support Python < 3.8. v2.13 was the last release supporting Python 3.7." 1 pyproject.toml:master.py-version = "3.8" 1 pyproject.toml:python_version = "3.8" 1 README.rst:lines of code and depend on Python (3.8+, or PyPy) and the C++ 2 README.rst:- Python 3.8+, and PyPy3 7.3 are supported with an implementation-agnostic 1 setup.cfg: Programming Language :: Python :: 3.8 1 setup.cfg:python_requires = >=3.8 1 setup.py:# TODO: use literals & overload (typing extensions or Python 3.8) 1 tests/CMakeLists.txt:if(NOT CMAKE_VERSION VERSION_LESS 3.8) 2 tests/constructor_stats.h:#if defined(PYPY_VERSION) 1 tests/env.py: doesn't work on CPython 3.8.0 with pytest==3.3.2 on Ubuntu 18.04 (pybind#2922). 1 tests/requirements.txt:build~=1.0; python_version>="3.8" 1 tests/requirements.txt:numpy~=1.21.5; platform_python_implementation!="PyPy" and python_version>="3.8" and python_version<"3.10" 1 tests/requirements.txt:numpy~=1.23.0; python_version=="3.8" and platform_python_implementation=="PyPy" 1 tests/test_buffers.py: env.PYPY, reason="PyPy 7.3.7 doesn't clear this anymore", strict=False 1 tests/test_builtin_casters.py: # Before Python 3.8, `PyLong_AsLong` does not pick up on `obj.__index__`, 2 tests/test_builtin_casters.py: if (3, 8) <= sys.version_info < (3, 10) and env.CPYTHON: 4 tests/test_builtin_casters.py: # TODO: PyPy 3.8 does not behave like CPython 3.8 here yet (7.3.7) 1 tests/test_callbacks.py: assert m.test_callback3(z.double) == "func(43) = 86" 2 tests/test_call_policies.cpp:#if !defined(PYPY_VERSION) 1 tests/test_chrono.py: diff = m.test_chrono_float_diff(43.789012, 1.123456) 1 tests/test_constants_and_functions.py: assert m.f3(86) == 89 1 tests/test_eigen_matrix.py: a_copy3[8, 1] = 11 1 tests/test_eigen_matrix.py: assert np.all(cornersc == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: assert np.all(cornersr == np.array([[1.0, 3], [7, 9]])) 1 tests/test_eigen_matrix.py: mymat = chol(np.array([[1.0, 2, 4], [2, 13, 23], [4, 23, 77]])) 1 tests/test_exceptions.py: if hasattr(pytest, unraisable): # Python >= 3.8 and pytest >= 6 2 tests/test_exceptions.py:@pytest.mark.xfail(env.PYPY, reason="Failure on PyPy 3.8 (7.3.7)", strict=False) 1 tests/test_factory_constructors.py: assert [i.alive() for i in cstats] == [13, 7] 1 tests/test_kwargs_and_defaults.cpp:#ifdef PYPY_VERSION 1 tests/test_local_bindings.py: assert i1.get3() == 8 1 tests/test_methods_and_attributes.cpp:#if !defined(PYPY_VERSION) 1 tests/test_numpy_array.py: a = np.arange(3 * 7 * 2) + 1 1 tests/test_numpy_array.py: assert str(excinfo.value) == "cannot reshape array of size 42 into shape (3,7,1)" 2 tests/test_numpy_array.py: assert x.shape == (3, 7, 2) 2 tests/test_numpy_array.py: m.reshape_tuple(a, (3, 7, 1)) 2 tests/test_numpy_array.py: x = m.reshape_tuple(a, (3, 7, 2)) 1 tests/test_numpy_vectorize.py: assert np.isclose(m.vectorized_func3(np.array(3 + 7j)), [6 + 14j]) 1 tests/test_pickling.cpp:#if !defined(PYPY_VERSION) 1 tests/test_pytypes.cpp:#if (defined(__APPLE__) && defined(__clang__)) || defined(PYPY_VERSION) 1 tests/test_smart_ptr.cpp: m.def("make_myobject3_1", []() { return new MyObject3(8); }); 1 tests/test_smart_ptr.py: assert cstats.values() == ["MyObject3[9]", "MyObject3[8]", "MyObject3[9]"] 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88]) 1 tests/test_stl_binders.py: assert v_int2 == m.VectorInt([0, 99, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 88, 4]) 1 tests/test_type_caster_pyobject_ptr.cpp:#if !defined(PYPY_VERSION) // It is not worth the trouble doing something special for PyPy. 1 tools/FindPythonLibsNew.cmake: set(PythonLibsNew_FIND_VERSION "3.8") 1 tools/JoinPaths.cmake:# https://docs.python.org/3.7/library/os.path.html#os.path.join 1 tools/pybind11NewTools.cmake: Python 3.8 REQUIRED COMPONENTS ${_pybind11_interp_component} ${_pybind11_dev_component} 1 tools/pybind11NewTools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: "3.12;3.11;3.10;3.9;3.8" 1 tools/pybind11Tools.cmake: if(NOT DEFINED PYPY_VERSION) 1 tools/pybind11Tools.cmake: message(STATUS "PYPY ${PYPY_VERSION} (Py ${PYTHON_VERSION})") 1 tools/pybind11Tools.cmake:# Python debug libraries expose slightly different objects before 3.8 1 tools/pybind11Tools.cmake: set(PYPY_VERSION ```

Skylion007 · 2024-07-30T02:36:10Z

pyproject.toml

@@ -30,7 +30,7 @@ ignore_missing_imports = true


 [tool.pylint]
-master.py-version = "3.7"
+master.py-version = "3.8"


ruff has a Python version too, right? otherwise UP036 would complain above

Yeah, it does: target-version = "py37"

This will go away if we move to scikit-build-core after 0.10 comes out. ;)

Thanks for catching this @Skylion007!

Fixed with commit 49580cf

That made me realize we need to add something like this to the list of useful git grep commands, for the next time when we want to drop a Python version:

git grep -I '[^0-9]37[^0-9]' git grep -I '[^0-9]38[^0-9]'

It will also go away if define the minimum python version in pyproject.toml standard location instead of setup.cfg :)

…kylion007

…kylion007

rwgk · 2024-07-30T16:18:29Z

Thanks @Skylion007 !

Ralf W. Grosse-Kunstleve added 2 commits June 24, 2024 09:59

First pass updating misc files, informed by https://github.com/pybind…

83b5ddc

…/pybind11/pull/5177/commits

Remove jobs using silkeh/clang and gcc docker containers that come wi…

2f24de5

…th Python 3.7

Ralf W. Grosse-Kunstleve added 2 commits June 24, 2024 13:32

Add silkeh/clang:17-bookworm

e4279a2

Add job using GCC 7

518515a

Ralf W. Grosse-Kunstleve added 10 commits June 24, 2024 15:22

Revert "Add job using GCC 7"

b339dc4

This reverts commit 518515a.

Try running in ubuntu-18.04 container under ubuntu-latest (to get GCC 7)

b203a29

Fix - vs : mixup.

b1c4304

This reverts commit b1c4304.

2324df0

Revert "Try running in ubuntu:18.04 container under ubuntu-latest (to get GCC 7)" This reverts commit b203a29.

git grep 0x03080000 cleanup.

af537e7

git grep -I -E '3\.7' cleanup.

bb5b9d1

Removes two changes made under pybind#3702

Revert "git grep -I -E '3\.7' cleanup."

551740d

This reverts commit bb5b9d1.

.github/workflows/configure.yml: Change from Python 3.7 to 3.8

e10d268

Misc cleanup pass

d255d63

Ralf W. Grosse-Kunstleve added 2 commits July 27, 2024 19:48

Merge branch 'master' into rm_py37

5de1214

rwgk marked this pull request as ready for review July 28, 2024 17:45

rwgk requested a review from henryiii as a code owner July 28, 2024 17:45

Merge branch 'master' into rm_py37

64a7b44

Skylion007 reviewed Jul 30, 2024

View reviewed changes

Change [tool.ruff] target-version to "py38", as suggested by @S…

49580cf

…kylion007

Skylion007 approved these changes Jul 30, 2024

View reviewed changes

rwgk merged commit 7233072 into pybind:master Jul 30, 2024
78 checks passed

rwgk deleted the rm_py37 branch July 30, 2024 16:18

github-actions bot added the needs changelog Possibly needs a changelog entry label Jul 30, 2024

henryiii removed the needs changelog Possibly needs a changelog entry label Aug 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: remove Python 3.7 support #5191

feat: remove Python 3.7 support #5191

rwgk commented Jun 24, 2024 •

edited

Loading

rwgk commented Jun 24, 2024 •

edited

Loading

rwgk commented Jun 24, 2024 •

edited

Loading

henryiii commented Jun 26, 2024

rwgk commented Jun 26, 2024

henryiii commented Jun 26, 2024

rwgk commented Jun 26, 2024

henryiii commented Jun 27, 2024

rwgk commented Jun 27, 2024

henryiii commented Jun 27, 2024

rwgk commented Jun 28, 2024

Skylion007 Jul 30, 2024

Skylion007 Jul 30, 2024

henryiii Jul 30, 2024

rwgk Jul 30, 2024

Skylion007 Jul 30, 2024

rwgk commented Jul 30, 2024

feat: remove Python 3.7 support #5191

feat: remove Python 3.7 support #5191

Conversation

rwgk commented Jun 24, 2024 • edited Loading

Description

Suggested changelog entry:

rwgk commented Jun 24, 2024 • edited Loading

rwgk commented Jun 24, 2024 • edited Loading

henryiii commented Jun 26, 2024

rwgk commented Jun 26, 2024

henryiii commented Jun 26, 2024

rwgk commented Jun 26, 2024

henryiii commented Jun 27, 2024

rwgk commented Jun 27, 2024

henryiii commented Jun 27, 2024

rwgk commented Jun 28, 2024

Skylion007 Jul 30, 2024

Choose a reason for hiding this comment

Skylion007 Jul 30, 2024

Choose a reason for hiding this comment

henryiii Jul 30, 2024

Choose a reason for hiding this comment

rwgk Jul 30, 2024

Choose a reason for hiding this comment

Skylion007 Jul 30, 2024

Choose a reason for hiding this comment

rwgk commented Jul 30, 2024

rwgk commented Jun 24, 2024 •

edited

Loading

rwgk commented Jun 24, 2024 •

edited

Loading

rwgk commented Jun 24, 2024 •

edited

Loading